Modeling Illumination Variation with Spherical Harmonics

Illumination can have a significant impact on the appearance of surfaces, as the patterns of shading, specularities and shadows change. For instance, some images of a face under different lighting conditions are shown in figure 1. Differences in lighting can often play a much greater role in image variability of human faces than differences between individual people. Lighting designers in movies can often set the mood of a scene with carefully chosen lighting. To achieve a sinister effect, for instance, one can use illumination from below the subject—a sharp contrast to most natural indoor or outdoor scenes where the dominant light sources are above the person. Characterizing the variability in appearance with lighting is a fundamental problem in many areas of computer vision, face modeling, and computer graphics. One of the great challenges of computer vision is to produce systems that can work in uncontrolled environments. To be robust, recognition systems must be able to work oudoors in a lighting-insensitive manner. In computer graphics, the challenge is to be able to efficiently create the visual appearance of a scene under realistic, possibly changing, illumination. At first glance, modeling the variation with lighting may seem intractable. For instance, a video projector can illuminate an object like a face with essentially any pattern. In this chapter, we will stay away from such extreme examples, making a set of assumptions that are approximately true in many common situations. One assumption we make is that illumination is distant. By this, we mean that the direction to, and intensity of, the light sources is approximately the same throughout the region of interest. This explicitly rules out cases like slide projectors. This is a reasonably good approximation in outdoor scenes, where the sky can be assumed to be far away. It is also fairly accurate in many indoor environments, where the light sources can be considered much further away relative to the size of the object. Even under the assumption of distant lighting, the variations may seem intractable. The illumination can come from any incident direction, and can be composed of multiple illuminants including localized light sources like sunlight and broad area distributions like skylight. In the general case, we would need to model the intensity from each of infinitely many incident lighting directions. Thus, the space we are dealing with appears to be infinite-dimensional. By contrast, a number of other causes of appearance variation are …